System and method for balancing video encoding tasks between multiple processors

ABSTRACT

System and method for balancing video encoding tasks between multiple processors. The method may include receiving a real time video stream, performing picture level and upper processing on a main processor, executing a macroblock loop in parallel on a main processor and a co-processor, wherein executing includes processing a first group of video encoding tasks on the main processor and processing a second group of video encoding tasks on the co-processor, and outputting an encoded version of the real time broadcast. The method may be implemented on a system that includes a main processor, a co-processor, and an interface to receive the real time video stream, each coupled to one or more buses. The encoding may be performed according to the well known Moving Pictures Experts Group (MPEG) standards.

RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.60/254,825 filed Dec. 11, 2000.

FIELD OF THE INVENTION

The invention relates generally to encoding digital video, and morespecifically, to encoding digital video according to the Moving PicturesExpert Group (MPEG) standard.

BACKGROUND

Video compression is becoming a requirement of many electronic devices.Video compression is an important component in devices such as personalcomputers, digital cameras, digital video cameras or camcorders, andpersonal video recorders (PVRs), also known as digital video recorders.To better understand how an why compression is becoming morecommonplace, review of the history of how consumers view and usetelevision signals is instructive.

Historically, television signals were broadcast from a transmitter andreceived and displayed immediately upon receipt on a televisionreceiver. As television evolved, cable television became widespread.However, with the advent of cable television, viewers still watchedtelevision shows as they were broadcast. With the introduction of thevideo cassette recorder (VCR), television viewers were able to recordtelevision broadcasts to be viewed at a time after the originalbroadcast of the particular television program. This provided televisionviewers a greater freedom to enjoy the television broadcasts at theirconvenience.

Relatively recently, PVRs had been made available to consumers. PVRsincluding the functionality provided by TiVo, Inc. of Alviso, Calif.,and Replay TV/Sonicblue, Inc. of Santa Clara, Calif., now allow thetelevision viewer even more options as to how and when a televisionprogram or other video content will be viewed. Among other things, PVRsallow a television viewer to pause a live broadcast of a program while,for example, answering the telephone or attending to some task in theviewer's home. In addition, PVRs may include technology which recordsspecific programs selected by a viewer from a program guide and may alsoinclude smart technology which automatically records programs meetingcriteria selected by a viewer while the viewer is not available or notwatching television. Generally, a PVR must receive a televisionbroadcast and store it in a format which later will be displayed to thetelevision viewer. Such storage must also occur in real time while theviewing of a currently live broadcast program is paused. To efficientlystore broadcast programs, programs are compressed prior to storage.Similarly, other personal electronic devices compress sequences of realtime images before storing them as video.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an environment in which one embodiment of theinvention executes.

FIG. 2 illustrates a flow of actions taken according to an embodiment ofthe invention described herein.

FIG. 3 illustrates the flow of execution of tasks in video encodingallocated between a co-processor and a main processor according to oneembodiment of the invention described herein.

FIG. 4 illustrates the flow of execution of tasks in video encodingallocated between a main processor and a co-processor according to oneembodiment of the invention described herein.

FIG. 5 illustrates the timing of tasks achieved when encoding digitalvideo according to one embodiment of the invention described herein.

FIG. 6 illustrates the tasks involved with encoding digital video andtheir allocation between a main processor and a co-processor accordingto one embodiment of the invention described herein.

FIG. 7 illustrates a conceptual diagram of what data is stored invarious memory locations during video encoding processing according toan embodiment of the systems and methods described herein.

DETAILED DESCRIPTION

Personal video recorders (PVRs) receive television broadcasts in oneformat and encode the television broadcasts in real time in a secondformat for storage on a hard disk drive or other storage device withinthe PVR. The broadcast material may be encoded according to the wellknown Moving Picture Expert Group (MPEG) standards before it is storedon the hard disk drive. The MPEG family of standards is entitled “Codingof Moving Pictures and Audio” and is a set of evolving standards,including MPEG-1, MPEG-2, MPEG-4, MPEG-7 and MPEG-21. The MPEG-1, MPEG-2and MPEG-4 standards are available from the International Organizationfor Standardization of Geneva, Switzerland as International StandardsOrganization/International Electrotechnical Commission (ISO/IEC) 11172,ISO/IEC 13818, and ISO/IEC 14496, respectively. Similarly, the materialstored on the storage device must be processed, including unencoding,when the stored broadcast material is retrieved for display to a viewer.It is the real time encoding required of a PVR that the systems andmethods described herein address. In addition, the methods describedherein may be incorporated into any devices that require real time videocompression, including, for example, digital cameras, digital videorecorders or camcorders, digital versatile disk (DVD) recorders, compactdisk (CD) recorders, personal computers, and the like.

FIG. 1 illustrates an environment in which one embodiment of theinvention executes. In one embodiment, personal video recorder (PVR) 100receives a real time video stream as a television broadcast stream froma hardwired connection such as network 156. In this embodiment, the realtime video stream may originate from broadcast server 158. In oneembodiment, network 156 may be the Internet or other widely accessiblewide area network (WAN). Similarly, PVR 100 may receive a real timevideo stream as a television broadcast signal that originated from atelevision broadcast transmitter or other wireless communication device,such as broadcast transmitter 142. PVR 100 may store broadcast materialfrom the broadcast stream and/or broadcast signal on a storage device,such as storage device 134 for later broadcast to a user via a displayor television monitor such as display 150.

In one embodiment, the personal video recorder may includemultiprocessor 110, storage device 134, input controller 136, andnetwork interface 138 coupled for communication with each other via bus140. In addition, memory 132 may be coupled directly to multiprocessor110 via memory bus 130. In one embodiment, memory 132 may be any form ofrandom access memory (RAM). In one embodiment, input controller 136 mayreceive user input from a remote control device such as remote control154. Network interface 138 may be a cable modem or other device whichallows for the receipt of broadcast communications in any of a varietyof formats according to any of a variety of well known broadcaststandards, including digital broadcast, analog broadcast, and otherstandards. That is, network interface 138 may allow for the receipt ofbroadcast material via a hardwired connection and/or wirelessly in aformat promulgated by the International Telecommunications Union (ITU),the Advanced Television Systems Committee (ATSC), digital television(DTV), the Moving Pictures Expert Group (MPEG), high definitiontelevision (HDTV), and other well known formats. In various embodiments,the network interface may be coupled to a hardwired network such as acable television network or a digital subscriber line (DSL) network, anEthernet network, a twisted pair network, a fiber optic network such asa synchronous optical network (SONET) or other physically presentconnection. Similarly, broadcast interface 144 may allow for the receiptof a wireless broadcast in the form of microwave, satellite, radio wave,and the like from broadcast transmitter 142. In another embodiment, twoor more network interfaces and/or broadcast interfaces may be includedso that two or more kinds of broadcast signals and broadcast streams maybe received by PVR 100 over two or more broadcast media including thehardwired networks and wireless broadcasts discussed herein.

In one embodiment, storage device 134 is a hard disk drive. In otherembodiments storage device 134 may be any machine readable mediumincluding magnetic disk drives such as hard disks, optical disk drivessuch as readable and writeable digital versatile disks (DVD-RWs),magnetic tape, semiconductor devices such as electronically erasableprogrammable read only memory (EEPROM) devices and flash memory devices,etc. Although only one storage device is depicted, PVR 100 may have twoor more storage devices. In another embodiment, an external storagedevice may be coupled to PVR 100. In one embodiment, broadcast materialsuch as television programs, movies, video games, raw data, music,audio, and anything that may be broadcast and received via networkinterface 138 and/or broadcast interface 140 may be stored on thestorage device. In one embodiment, the broadcast material in the form ofa television, HDTV or other similar broadcast may be digitally processedand encoded in real time before it is stored on storage device 134. Itis multiprocessor 110 that processes and encodes the broadcast materialfor storage. In one embodiment, the storage device may includeinstructions which direct and control how multiprocessor 110 willprocess and encode the broadcast material. In one embodiment, themethods described herein may be implemented in software and stored onstorage device 134 as video processing software (VPS) 135.

In one embodiment, multiprocessor 110 may include a variety ofcomponents including main processor 112 and co-processor 114. Mainprocessor 112 may have its own private cache memory such as cache 113,and co-processor 114 may have a private cache memory such as cache 116.In one embodiment, main processor 112 may be considered a core processorhaving a very long instruction word (VLIW), and co-processor 114 may beconsidered a variable length encoding/decoding (VLx) processor. In oneembodiment, the main processor may have a VLIW word size of 64 bits.Also included in multiprocessor 110 is memory controller 118 whichallows for transmission of data to and from memory. 132 via memory bus130. In various embodiments, memory controller 118 may be used by adirect memory access (DMA) controller, such as DMA controller 117, toaccess memory external to the multiprocessor. DMA is a programmabledevice that controls the automatic data transfer between any two memoryblocks. Multiprocessor 110 may also include audio output interface 122and video output interface 124, which allow for the broadcast of videosignals and associated data from personal video recorder 100 to display150 and associated speakers 152. Main processor 112, co-processor 114,audio output interface 122 and video output interface 124 may be coupledfor communication with one anther via bus 126. It is via bus 126 thatmultiprocessor 110 communicates with other components of the personalvideo recorder via bus 140. In one embodiment, multiprocessor 110 may bea media processor such as the media accelerated processor for consumerappliances (MAP-CA) device available from Equator Technologies, Inc. ofCampbell, Calif. and may be the TriMedia TM32 system on a chip availablefrom TriMedia Technologies, Inc. of Milpitas, Calif. In anotherembodiment, each of the components within multiprocessor 110 may beseparately and individually included in PVR 100. In this embodiment, oneor more buses may be included in the PVR.

Although shown as implemented in PVR 100, the methods described hereinmay be implemented in any device requiring real time video encoding,including, for example, digital cameras, digital video recorder orcamcorders, DVD recorders, CD recorders, personal computers, and thelike.

FIG. 2 illustrates a flow of actions taken according to an embodiment ofthe invention described herein. In one embodiment, to take advantage ofa multiprocessor that includes a main processor and a co-processor, thetasks included in encoding a video signal as a digital video file forlater retrieval and viewing may be efficiently allocated between themain processor and the co-processor. In one embodiment, this encoding isaccomplished according to the MPEG-2 standard. Much of the terminologyused herein is well known in the art and is included in the MPEG-2standard. The tasks involved in video encoding include input loading, asshown in block 210. After the data has been input and loaded,preprocessing and prefiltering are performed, as shown in blocks 212 and214. Motion estimation may then be performed. In one embodiment, motionestimation may be broken down into two phases, Phase 1 and Phase 2, asshown in blocks 216 and 218. In one embodiment, motion estimation Phase1 includes top to top and bottom to bottom searching, and motionestimation Phase 2 includes top to bottom and bottom to top searching,as well as a final estimation decision. Motion estimation Phase 2 may becombined with motion compensation, as shown in block 218. The output ofmotion estimation phase two includes motion vectors. Forward discretecosine transform (FDCT) and quantization are then performed, as shown inblock 222. The output of block 222 is a quantized two-dimensional 8×8discrete cosine transform (DCT) matrix resulting from the FDCT. Theinverse discrete cosine transform and the inverse quantization are thencomputed, as shown in block 224. The result of the inverse discretecosine transform and inverse quantization computed in block 224 areadded to the result of the motion compensation performed in block 218,as shown in block 226. The result of this addition is saved into localdecoder output buffer 228. The data in local decoder output buffer 228is used as reference data by motion estimation Phase 2 in block 218 forprocessing other frames. This amounts to a feedback loop well known inthis kind of processing. Zig zag coding is then performed in block 230.Zig zag coding converts the two-dimensional 8×8 DCT coefficients matrixinto a one-dimensional array of 64 coefficients. Run and level coding isthen performed in block 242. Run and level coding converts theone-dimensional array of 64 coefficients into a set of non-zerocoefficients separated by some zero coefficients. The set of non-zerocoefficients is called level and the number of zeros between eachnon-zero coefficients is called run. Each run/level pair and motionvectors from block 218 are encoded by variable length coding performedin block 244. Video code is stored in video code local buffer 246. Thestored video code results from the variable length coding of block 244,and the picture and upper level coding of block 250. The three steps of(1) run level coding, (2) variable length coding, and (3) copying thevideo code into-the video code local buffer may be thought of as a groupas variable length encoding, as shown in block 240. The result isthen-copied into video coder output buffer 252. To achieve increasedperformance of encoding video signals as a digital file according to theMPEG standard, the variable-length encoding tasks may be allocated to aspecialized co-processor and performed in parallel while the remainderof the tasks involved in encoding are performed on a main processor. Itis this parallel processing that provides increased efficiency andthroughput when encoding video to be stored as digital video in a PVR orother device according to the methods described herein.

FIG. 3 illustrates the flow of execution of tasks in video encodingallocated between a co-processor and a main processor according to oneembodiment of the invention described herein. Main processor 302 firstperforms picture level and upper processing as shown in block 306. Atthat point what will be referred to as macroblock loop 308 is entered.During macroblock loop 308 both co-processor 304 and main processor 302execute in concurrent, co-executing infinite loops. Main processor 302waits for the co-processor to finish the previous macroblock and for theDMA to transfer data into the local cache of the main processor and thenchecks the bit count of the data encoded by the co-processor, as shownin block 310. As is well known, according to the MPEG-2 standard, thebit count of one macroblock may not exceed a standard specified maximumnumber of bits. In one format, the standard specified maximum is 4,608bits. However, the standard also specifies an exception such that amaximum of two macroblocks in each horizontal row of macroblocks mayexceed this limitation. Preprocessing of the future frame, includingnoise reduction is achieved in block 312. The main processor thenperforms motion estimation Phase 1 on a future frame and motionestimation Phase 2 on a current frame, as shown in block 314. Then, theapplicable mode is selected, as shown in block 316. The mode willdetermine if the current macroblock is coded as intra mode or intermode. In intra mode, a forward discrete cosine transform (FDCT) will beapplied to the original pixels. In inter mode, FDCT will be applied tothe residual pixels after motion compensation. A FDCT is then executed,as shown in block 318. Rate control is then performed in block 320. Themain processor then performs forward quantization in block 322. Scanningfor zig zags is performed in block 324. The main processor then preparesfor processing the next macroblock, as shown in block 326. Suchpreparations may include, in one embodiment, setting up the appropriatememory descriptors of the next macroblock which will be used by the DMAcontroller. The flow of execution then loops back to block 310.

In co-processor 304, execution begins with waiting for data in aninfinite loop, as shown in block 330. Then, depending on the mode,execution continues. In one embodiment, there are three possible modesof execution. Mode 1, shown as reference number 332 is a bypass mode inwhich no variable length encoding preformed. The co-processor bypassesthe variable length encoding block 350. The main processor may use thismode to send coded picture level and upper data to the video code bufferin the main memory. As such, when Mode 1 has been selected, executionproceeds from block 330 to block 340 where the video code is written.Execution will then continue at block 360 at which time the currentlyprocessed data is written to the co-processor's cache, and executionresumes at block 330. In Mode 2, shown as reference number 334, normalvariable length encoding is achieved and this may be considered normalmode. As such, in Mode 2, VLE block 350 is executed. That is, afterblock 330, when in Mode 2, the macroblock header is encoded, as shown inblock 352, the motion vector is encoded, as shown in block 354, anddiscrete cosine transform (DCT) coefficients are encoded, as shown inblock 356. The processing of blocks 352, 354 and 356 are consideredvariable length encoding and may be thought of as VLE block 350. Uponcompletion of VLE block 350, the results are written to the co-processorcache, as shown in block 360. The flow of execution then continues atblock 330. If when in block 330 the co-processor determines that themain processor has selected Mode 3, shown as reference number 336,execution continues at block 338. Mode 3 may be referred to as theencode slice header code and macroblock mode. In this mode, theco-processor generates a slice header on top of an encoded macroblock.That is, the co-processor encodes the slice header, as shown in block338. Execution will then continue in the VLE block 350 and itsconstituent blocks as discussed above.

FIG. 4 illustrates the flow of execution of tasks in video encodingallocated between a main processor and a co-processor according to oneembodiment of the invention described herein. In this embodiment,various tasks involved with video encoding may be moved out of themacroblock to increase throughput and efficiency when encoding digitalvideo by freeing up motion estimation Phase 1 local memory cache whilethe main processor runs the macroblock encoding loop. The methodsdescribed herein may be used when the local cache of the main processoris not sufficiently large enough to handle the memory requirements ofestimation Phase 1. The disadvantage of this configuration is that theco-processor stays idle when main processor is working on motionestimation Phase 1. As such, the available execution time of theco-processor is reduced. In this embodiment, main processor 402 firstperforms picture level and upper processing in block 406 and thenperforms motion estimation Phase 1 on a future frame and motionestimation Phase 2 on the first two macroblocks, as shown in block 407before proceeding with the macroblock look 408. After processing,according to blocks 406 and 407, main processor 402 continues at block410. Concurrently, co-processor 404 will begin processing at block 430.Co-processor 404 sits idle while blocks 406 and 407 are being executedon main processor 402. Co-processor 404 waits for data from the mainprocessor, as shown in block 430.

In macroblock loop 408, main processor 402 waits for the co-processor tofinish processing the previous macroblock and for the DMA to transferdata into the local cache of the main processor and then checks the bitcount of the data encoded by the co-processor, as shown in block 410.Main processor 402 preprocesses one macroblock of a future frameincluding performing noise reduction, as shown in block 412. The mainprocessor then performs motion estimation Phase 2 on the current frame,as shown in block 414. The main processor then selects the particularmode, as shown in block 416. A forward discrete cosine transform is thenexecuted by the main processor, as shown in block 418. Rate control isthen performed, as shown in block 420. The main processor then performsforward quantization, as shown in block 422. A scan for zig zags is thenperformed, as shown in block 424. The main processor then performsinverse quantization, inverse discrete cosine transform, and adds thesetogether, as shown in block 426. The main processor then prepares forthe next macroblock, as shown in block 428. The flow of execution thencontinues at block 410.

With regard to the processing of co-processor 404, execution begins withwaiting for data in an infinite loop, as shown in block 430. Then,depending on the mode, execution continues. As discussed above, thereare three possible modes of execution in one embodiment. Mode 1, shownas reference number 432, is a bypass mode in which there is no variablelength encoding preformed. The co-processor bypasses the variable lengthencoding block 450. As such, when Mode 1 has been selected, executionproceeds from block 430 to block 440 where the video code is written.Execution will then continue at block 460 at which time the currentlyprocessed data is written to the cache, and execution resumes at block430. In Mode 2, shown as reference number 434, variable length encodingis achieved. Mode 2 may be considered normal mode. As such, in Mode 2,VLE block 450 is executed. That is, after block 430, when in Mode 2, themacroblock header is encoded, as shown in block 452, the motion vectoris encoded, as shown in block 454, and discrete cosine transform (DCT)coefficients are encoded, as shown in block 456. Upon completion of VLEblock 450, the results are written to the co-processor cache, as shownin block 460. The flow of execution then continues at block 430. Mode 3,shown as reference number 436, may be referred to as the encode sliceheader code on top of an encoded macroblock mode. In this mode, theco-processor encodes the slice header, as shown in block 438. Executionwill then continue in the VLE block 450 and its constituent blocks asdiscussed above.

FIG. 5 illustrates the timing of tasks achieved when encoding digitalvideo according to one embodiment of the invention described herein.When viewing the timing of each of the tasks involved in encodingdigital video, it becomes apparent that the tasks may be allocated amongtwo processors according to the methods described herein. For example,when a current macroblock referred to as MB(i), is being processed bythe main processor an earlier macroblock such as MB(i-5) may beprocessed by a co-processor. When the current macroblock MB(i) is beingloaded for motion estimation, as shown in the drawing as number 512, theprior macroblock MB(i-1) may be undergoing Phase 2 of motion estimation,shown by reference number 510. In one embodiment, the tasks involvedwith digital video encoding may be thought of as: motion estimationPhase 2, as shown by reference number 510; loading into memory thecurrent macroblock and associated reference data required for motionestimation, as shown by reference number 512; prefiltering the currentmacroblock of the future frame, as shown by reference number 514;loading the filter reference data into memory, as shown by referencenumber 516; transferring the decoder/filter output to main memory, asshown by reference number 518; loading the current macroblock (MB) andthe motion compensation (MC) reference block base of the motion vectorresulting from motion estimation Phase 2, as shown by reference number520; encoding the current macroblock including performing mode decision,motion compensation, forward discrete cosine transform, rate control,forward quantization, inverse quantization and inverse discrete cosinetransform, as shown by reference number 522; sending the macroblock tothe co-processor, as shown by reference number 524; and the co-processorcoding the macroblock, as shown by reference number 526. Review of FIG.5 shows that each of these tasks may be achieved on differentmacroblocks at the same time to achieve greater encoding throughput byparallel processing. For example, when at time T₁ a current macroblockis loaded into memory, as shown at the intersection of T₁ with referencenumber 512, the co-processor may be working on an earlier macroblocksuch as MB(i-5) while the main processor is performing other tasks onother macroblocks and particularly while the current macroblock MB(i) isbeing loaded into memory. These benefits may be seen more clearly withreference to the next drawing.

FIG. 6 illustrates the tasks involved with encoding digital video andtheir allocation between a main processor and a co-processor accordingto one embodiment of the invention described herein. Main processor 602may encode picture level data as shown in block 610. The result of thisencoding is then stored in memory via a memory access, as shown in block612. During this main processor processing of block 610, co-processor604 sits idle and waits for picture data to be stored in memory, asshown in block 614. The co-processor, upon receipt of the picture datafrom memory, passes the picture code to its output code buffer, as shownin block 614. The-output of the co-processor is the code size of thepicture level and upper code, as shown in block 620. While theco-processor is processing the picture code in block 614, the mainprocessor is encoding macroblock MB(0) with a slice flag, as shown inblock 616. Encoding MB(0) involves the processing described aboveregarding block 522 of FIG. 5. After the main processor encodes themacroblock with a slice flag, as shown in block 616, the main processoraccesses memory as shown in block 624 to provide the result of theencoding to the co-processor. The co-processor waits for the result ofthe encoding of macroblock MB(0) with a slice flag in block 616 and thenencodes macroblock MB(0), as shown in block 626. While the co-processorencodes macroblock MB(0) in block 626, the main processor encodesmacroblock MB(1) and predicts the size of macroblock MB(0), as shown inblock 618. After the main processor encodes macroblock MB(1) andpredicts the size of macroblock MB(0), as shown in block 618, the mainprocessor accesses memory, as shown by block 628. The co-processor waitsfor macroblock MB(1) from the main processor and then encodes macroblockMB(1) as shown in block 634. That is, the co-processor retrieves frommemory the result of the encoding and predicting of the main processorfrom block 618. While the co-processor is processing or encodingmacroblock MB(1), as shown in block 634, the main processor, afterwaiting for picture and upper level data encoding to be completed by theco-processor, then encodes macroblock MB(2) and predicts the size ofmacroblock MB(1), as shown in block 622. The main processor receives acode size of macroblock MB(0) from the co-processor after the encodingby the co-processor achieved in block 626. If the bit count ofmacroblock MB(0) returned by co-processor is over the MPEG-2 standardimposed size limitation, such as, for example 4,608 bits, the mainprocessor will set the quantization step size to a maximum value for thewhole row to guarantee that there will be no additional oversizedblocks. Then, the main processor re-encodes MB(2) before sending it tothe co-processor and encodes macroblock MB(3), as shown in block 632. Asa result, both macroblock MB(0) and macroblock MB(1) may be bigger than4,608 bits. Because the MPEG-2 standard allows at most two bigmacroblocks per horizontal row, the generated video code adheres to theMPEG-2 video stream standard. The co-processor also writes the bit countinto the main processor's local cache. In one embodiment, this is donedirectly. All other data transfers are achieved via the DMA controller.Parallel processing on the main processor and co-processor thencontinues in a similar fashion until encoding is completed.

FIG. 7 illustrates a conceptual diagram of what data is stored invarious memory locations during video encoding processing according toan embodiment of the systems and methods described herein. The mainprocessor may include main processor cache 704, and the co-processor mayinclude co-processor cache 706. Main memory 702 is also used during thedigital video encoding described herein. In one embodiment, an encodedframe is stored in main memory as EFRAME (i-1) 710. The encoded frame isretrieved by the main processor and stored in main processor cache 704after processing as reference macroblock (RMB) 720. After someprocessing, the main processor creates a current macroblock and alsoretrieves data from main memory 702 in the form of a current frameCFRAME(i) 712. Current macroblock (CMB) 722 is then processed and theresulting data is passed to the discrete cosine transform/forwardquantization step, as shown in block 724. Results of the discrete cosinetransform/forward quantization step are passed to the inverse discretecosine transform/inverse quantization (IDCT/IQ) step, as shown in block726. In addition, the results of the discrete cosine transform/forwardquantization (DCT/FQ) step shown in block 724 are also passed from themain processor cache 704 to the co-processor cache 706. The co-processorthen performs a variable length encoding at block 730 to produce videocode 740. While working on processing variable length encoding (VLE), asshown by block 730, the co-processor stores data in co-processor cache706. This is the same VLE referred to in blocks 350 and 450 discussedabove regarding FIG. 3 and FIG. 4, respectively. Upon completion of thisprocessing, the video code is then written to main memory, as shown inblock 740. Going back to block 726, upon completion of the inversediscrete cosine transform, the main processor writes the results intomain memory 702 as encoded frame EFRAME (i), as shown in block 714.

In the foregoing specification, the invention has been described withreference to specific embodiments thereof. It will, however, be evidentthat various modifications and changes can be made thereto withoutdeparting from the broader spirit and scope of the invention as setforth in the appended claims. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense.

1. A system comprising: an interface coupled to a bus to receive a realtime video stream; a main processor coupled to the bus, the mainprocessor to process a first group of video encoding tasks comprisingthose video encoding tasks not including variable length encodinginvolved with encoding the real time video stream; a co-processorcoupled to the bus, the co-processor to process a second group of videoencoding tasks including variable length encoding tasks involved withencoding the real time video stream. 2-39. (canceled)